Complexity and Performance Assessment for Data Fusion Systems

Demonstrated, documented performance is a prerequisite before a data fusion system may be deployed. Developers and users must be confident about fusion system performance across the full range of operating conditions and scenarios the system is anticipated to encounter. We report on an approach to multi-sensor data fusion performance characterization which systematically explores system performance and quantifies performance degradation at and beyond the limits of the intended application scenarios. A quantitative characterization of the complexity of test scenarios supports our experimental approach to performance assessment. Scenario complexity characterization directs creation and systematic variation of test scenarios and facilitates efficient exploration of the range of relevant fusion scenarios. Data Fusion performance metrics measure the quality of the track picture produced by the data fusion solution and the correctness of the intermediate constituent processing steps. Track picture quality is measured by the accuracy, precision, consistency, and completeness of the fused track picture. Constituent metrics function as “built-in-test” procedures for critical processing steps and reveal causes for sub-optimal performance. They indicate when the fusion system under test operates on a scenario which approaches the limits of its capabilities. We successfully applied the complexity and performance measures described in this paper to the development and validation of the Rotorcraft Pilot’s Associate (RPA) Level 1 Sensor Fusion component.